Measure Valhalla/Panama layout authority and mask-native Rust execution - #24

…) alone lies

R4 asks whether the V3 register's three carvings (6x(u8:u8) rails, 4x(u8:u8:u8)
triplets, 3x(u8:u8:u8:u8) quads) dodge the 8-byte array-flattening cliff R2
measured on monolithic 12/16-byte value classes.

They do not, and the nested spelling is strictly worse:

  - Pair/Triplet/Quad alone are flat in all three array kinds.
  - Reg12AsRails/Triplets/Quads and Facet16As* are all false, identical to the
    monolithic control Reg12Flat.
  - Nesting costs flatness even UNDER the budget (Nest7 false vs Flat7 true),
    because a record component is nullable by default and stored in its
    nullable flat layout (Pair 2->4, Quad 4->8).
  - That mechanism is confirmed, not assumed: @NullRestricted flips all three
    predicted failures (Nest7NR, Nest8AsQuadsNR, Nest6AsPairsNR) false->true.
  - Removing the inflation still does not rescue 12/16 B. The cliff is on
    total payload.

Consequence: the carving is sound as SoA and only as SoA -- N parallel rail
arrays, never one Facet[].

Group F (the word-aligned family) then produced a non-monotone row that looked
like good news: Four8AsTwo8, a 32-byte record, reports flat. The VM's own
element size is 8. Its @NullRestricted Two8 components are themselves
non-flattenable, so each is stored as a REFERENCE -- a flat array of pointers,
the opposite of the property being sought. Nest8Single shows the same hazard
inverted: an 8-byte payload at element size 16.

So isFlatArray() alone is not a sufficient test. R4-observed.txt now pins the
VM element sizes beside every boolean, from -XX:+PrintFlatArrayLayout.
Answering the 2x8-grouping question from the boolean alone would have shipped
a false positive.

R5 measures what a classid-dependent layout costs. Neither mechanism can
express one: a VarHandle is bound to its path at construction, a value class is
a static type, so the carving choice is a Java-side switch in every design.
Over 65,536 rows, projecting straight from the MemorySegment allocates 800 B
total (0.01 B/row); hydrating a 16-byte Facet costs 32-104 B/row, varying by
run because escape analysis is best-effort and nothing in the source chooses
whether it fires. The spread is the finding, not noise.

Also corrects java/README.md's [restricted]-warning count from six to seven.
The board has said seven since 2026-08-17 (STATUS_BOARD D-LGJ-D and D-LGJ-W3,
LATEST_STATE), with the more accurate characterisation; this file was the only
place carrying the stale count. Recorded as a dated correction in place.

Records the R4/R5 measurements and the division-of-authority reading, and
notes that 6828f4a landed the code without this entry -- a same-commit rule
break, recorded rather than back-dated.

…or a flag

Answers whether the 8-byte cliff is something a later JDK or a VM flag repeals.

Not a version gap: R4/R5/R6 all run on 27-jep401ea3+1-1, the JEP 401 EA. These
already are the Java 27 numbers.

Not a tunable: forcing all five flattening flags produces output byte-identical
to the default run. Worth doing rather than assuming, since UseArrayFlattening
and UseFieldFlattening are false by DEFAULT in this build.

By design: JEP 401 states it -- a flattened reference must be read and written
atomically, which on common hardware caps mutable flattened fields at 64 bits.

R6 then tests the exemption the JEP names ('the fields of a value class do not
have this atomicity limitation'), which R4 never covered. Measured: a 12-byte
Reg12 as a value-class field is REGULAR 4/4 -- a reference, not flattened --
while the 4-byte control flattens. Whether that is an EA gap or a further
constraint is left as an open item, not claimed.

The durable point does not depend on resolving it: the exemption is for FIELDS
and SoA lanes are ARRAYS, whose elements are mutable by definition. The JEP's
own speculative 128-bit note would move the cliff to 16 B -- the 12-byte
register would fit, the 512-byte row would not.

The endgame claim measured falsifiably: 10^9 group projections (classid ->
carving dispatch -> project straight from the MemorySegment, no element type
ever created) allocate 960 B TOTAL, byte-identical across three runs, at
369-439 M ops/s single-threaded (2.3-2.7 ns/op).

The decisive comparison is R5's own scaling: 65,536 ops allocated 800 B, so
operations grew 15,000x while allocation grew 160 B -- fixed scaffolding, not
per-op cost. Any per-op survivor of even 1 B would have shown as a gigabyte.
The hydrating alternative at the same op shape (32-104 B/row per R5) would be
32-104 GB at this scale.

2.3 ns/op is memory-latency scale because there is no FFI in the loop and no
object either: the zero-copy pointer IS the MemorySegment plus offset
arithmetic, and Java's layout authority never engages because it is never
handed a type.

…ns outright

R7 proved Java projection allocates nothing and deliberately proved nothing
about Java-vs-Rust speed (no FFI, no Rust in its loop). R8 measures that, five
arms, brutally symmetric: same 1 MiB filled by the SAME native r8_fill for
every arm, same classid distribution, same 10^9-op accounting, same checksum.
Checksum equality across arms -- including a standalone Rust process with no
JVM -- is the proof they did the same work on the same bytes. Arm E further
asserts its mask popcounts equal the Java partition scan's counts.

All Rust arms at -Ctarget-cpu=x86-64-v4 (host has avx512f/bw/dq/vl/vbmi/ifma;
ndarray's own config pins v3 and this crate sits outside it, so v4 is set
explicitly -- that is what selects the simd_avx512 dispatch arm). Built with
CARGO_PROFILE_RELEASE_DEBUG=0: 23 s, 118M target.

PART 1, period-4 classid (the control):
  B bulk FFI -> generic Rust  2417-2479 M ops/s, indistinguishable from the
    standalone Rust process. ONE BULK FFI CROSSING COSTS NOTHING MEASURABLE.
  D monomorphic kernels       2314-2337 -- D > B is FALSIFIED. A predictable
    pattern is already specialized by the branch predictor.
  A Java in-process           266-281 (~9x behind Rust, same loop)
  C FFI PER PROJECTION        74-81 = ~12.9 ns/op, ~31x slower than B. The
    anti-JNI rule with a number on it.

PART 2, random classid (where dispatch actually costs):
  B' generic Rust  834-841 M ops/s -- a 2.9x collapse; the dispatch cost was
     always there, the predictor was paying it.
  D' Java index-list partition -> monomorphic kernels  3959-4014 (4.8x)
  E' ndarray::simd bitmasks -> mask-bit sweep          3892-3908 (4.65x)

THE RESULT THAT MATTERS: E' exists because D' cheats -- an index list is a
materialized population, forbidden as internal currency by the mask-native
law. E' was expected to PAY for obeying the law. It does not. It costs 2-3% in
the sweep and wins decisively on the step that differentiates them:

  Java scalar scan -> index lists              41.01 ms
  ndarray::simd::eq_u32_strided_to_mask -> masks 2.49 ms   (16.5x faster)

  break-even for the mask build : ~10 passes
  break-even for the Java scan  : ~167 passes

The lawful currency is 16x cheaper to create and pays for itself 16x sooner.
Obeying the law is the fast path, not a tax on it.

Not "Java is faster than Rust": the winning kernels ARE Rust and so is the
mask builder. The win is specialization PLACEMENT. Part 1 is the control that
keeps part 2 honest -- when dispatch is predictable, partitioning buys nothing.

Also in this commit, both from operator review of R7:
  - R7's prose quoted a 369-439 M ops/s range its own pinned runs contradicted
    (1.76-3.48 s across three runs). Throughput is now explicitly NOT banked;
    only the exact 960 B allocation is. The contradiction is recorded in the
    observed file rather than papered over.
  - R7's "memory-latency scale" was the wrong regime -- the working set is
    ~1 MiB and cache-resident. Corrected to hot-cache load-and-shift scale.
  - R5's "neither Panama nor Valhalla can select a layout by runtime classid"
    was broader than the measured fact. Panama CAN construct or choose a
    MemoryLayout at runtime; what it cannot do is make an already-bound
    VarHandle reinterpret its path per row. Tightened in both the README and
    the reproducer source to the durable statement: runtime classid requires
    descriptor/accessor dispatch.